In recent years, an imaging apparatus is widely applied for an in-vehicle camera which is mounted on a vehicle.
Here, types of the in-vehicle cameras include, for example, a rear-view camera which helps checking behind when parking a vehicle, a side-view camera which helps checking dead angles of sides of a vehicle when parking a vehicle in tandem, and a front-view camera which helps checking left and right when, for example, entering into an intersection. Also, other types of the in-vehicle cameras include, for example, a camera for a drive-recorder which takes and records images in the direction of travel of the vehicle all the time, and a camera for front sensing which is used for recognition of objects such as white lines of a road, median strips, road signs, traffic signals, display boards, etc.
It is required for an in-vehicle camera to obtain an image (an appropriate image) in which a subject (object of shooting) is recognizable even in the case where the image is taken in a dark place such as on a road without a street light for night or in a tunnel without illumination. In order to obtain an appropriate image, for an in-vehicle camera, various countermeasures are taken including a longer exposure time of an imaging element (image sensor), an increased analog gain of an imaging element, a brighter lens with a smaller F value, or the like.
Here, the longer exposure time is for increasing an amount of received light by increasing the time the acceptance surface of the imaging element is irradiated with light. Also, the increased analog gain is for making an output image look brighter by increasing an amplitude of a detected signal. Furthermore, the smaller F value of the lens is for making the lens brighter, because, in general, the smaller the F value is, the brighter the lens becomes.
With these countermeasures, it becomes possible for an in-vehicle camera to make a dark area of an image look brighter, thereby making a subject in a dark area of an image easy for recognition even if it is taken at night.
On the other hand, when an image includes a bright area and a dark area as in a downtown at night, in the case where the subject is a light emitter such as a street light, a head lamp of an oncoming car, a tail lamp of a vehicle ahead, a blinker, a traffic signal, or the like, sometimes, a so-called blown-out highlight appears in the image.
In the case where a blown-out highlight appears in an image, it becomes difficult to discriminate color of the subject, thereby making it difficult to discriminate between a tail lamp and a blinker, between a street lamp and a traffic signal, or between colors of traffic signals.
Also, a blown-out highlight in an image of the subject appears in the case where the light of the head lamp of the vehicle is emitted on the subject.
Here, in the case where the subject on which the light of the head lamp is emitted is a vehicle ahead, there is a problem that it is difficult to determine whether or not the tail lamp of the vehicle ahead is lit, because of the blown-out highlight of the image of the vehicle ahead.
It can be seen that in order to suppress the blown-out highlight of the image of the subject, various countermeasures for an in-vehicle camera can be considered including, contrary to the settings as described above for obtaining an appropriate image in a dark place, a shorter exposure time of an imaging element, a decreased analog gain of an imaging element, a darker lens with a bigger F value, or the like.
However, with these countermeasures, the entire image becomes dark, and therefore, in the case where these countermeasures are used for an in-vehicle camera, especially for a rear-view camera, a front-view camera, or a front sensing camera, it becomes difficult to recognize, for example, a condition of a dark road surface. The reason for the difficulty is that it is difficult to include the luminance range of the darkness of the road surface and the brightness of the light source within a dynamic range of the imaging element.
Meanwhile, in recent years, there are methods such as HDR (High Dynamic Range), WDR (Wide Dynamic Range), or the like, for expanding the dynamic range of the imaging element.
The HDR and the WDR are techniques for making a dynamic range in an image bigger than normal by obtaining an image for which longer exposure time is applied to the imaging element and an image for which shorter exposure time is applied to the imaging element, combining the two images, and creating one image.
For an imaging element to obtain two images with different exposure times, there is a method in which, using one pixel, an image for which longer exposure time is applied and an image for which shorter exposure time is applied are obtained at different timings by one pixel of the image elements. Also, for an imaging element to obtain two images with different exposure times, there is another method in which images with different exposure times are obtained by providing two areas in one pixel: an area where longer exposure time is applied, and another area where shorter exposure time is applied; thereby images with different exposure times can be obtained in real time.
In the case where two images with different exposure times are obtained and combined for creating an image, image data for a dark image area such as a dark road at night which is cut from the image with longer exposure time and image data for a bright image area such as a bright sky which is cut from the image with shorter exposure time are combined for creating one image.
By using HDR or WDR, a blown-out highlight of a bright area and an underexposure of a dark area can be suppressed, and an image at a time of low illumination can be obtained as a bright and clear image by the in-vehicle camera.
However, when combining two images with different exposure times, it is difficult to accurately determine light and shade of the bright area and the dark area, and divide the image areas. In other words, when combining two images using HDR or WDR, it is easy to separate a relatively large bright area from a dark image, but it is difficult to identify a small bright area such as a point light source and separate it from the dark image. Therefore, in the case where the subject whose blown-out highlight should be suppressed is a luminous point or the like, a sufficient effect cannot be obtained from HDR or WDR.
Also, a blown-out highlight in the case where a luminous point is a subject occurs because each color signal value of RGB (Red, Green, Blue) becomes closer to a saturation value, thereby a value difference becomes infinitesimal and color of an emission source becomes thinner and closer to white color.
Based on the above, in the case where a luminous point is a subject, a method can be considered in which the color of the luminous point appears by increasing the chroma of the entire image data in a state where the color is thinner.
However, in this case, because the chroma other than the luminous point is also increased, not only the overall color of the image becomes dark creating an uncomfortable feeling, but also the chroma of the black area increases. Therefore, the obtained image includes increased color noise created by increased signals of the black area containing a lot of noise, thereby S/N (Signal/Noise) ratio becomes lower.
It should be noted that, in order to form an image with little collapse color and little color saturation even when a subject has a big difference between the light and dark, a technique is proposed in which color component signals in each corresponding pixel position are compared with each other between the N original images and the color component signal which produces a higher chroma is set as a color component signal of the output image (see, for example, Patent Document 1).
However, in the prior art, a process is needed in which, similar to HDR or WDR, two images with different exposure times need to be obtained before combining the obtained images.
Also, the prior art cannot solve a problem of difficulty in recognizing the color of the subject image with the blown-out highlight described above because an image with a shorter exposure time is used for the blown-out highlight area in an image with a longer exposure time.
Also, the prior art has a problem, for example, in which because a black area with no chroma ends up having a chroma due to low illumination noise, when the chroma is emphasized, colors are attached to the black area, generating an image different from the actual state.
It is a general object of the present invention to provide an image processing apparatus which is capable of reproducing the color of a bright subject even in the case where the bright subject is included in a dark image.